1. Field of the Invention
This invention relates to an image sensing and processing device which is highly suited for a video camera or the like and is arranged to detect a moving part of an image plane, to set an object tracing area and a distance measuring area and to perform automatic focusing and automatic image stabilization control.
2. Description of the Related Art
In the image sensing optical apparatuses of varied kinds including video cameras, etc., irrespective as to whether they are adapted for industrial instrumentation, or consumer appliances, shaking of an image not only hinders easy image sighting but also degrades image recognizing accuracy. In the case of the video camera, for example, the camera is often operated while the operator is walking or while the camera is on a moving vehicle. In such a case, it is inevitable to have a sensed image shaken by a shake of the camera according to the photographing conditions or the object to be photographed.
To solve this problem, there have been proposed image shake detecting devices adopting varied methods. In one of such methods, the movement of the camera is physically detected by means of an acceleration sensor (an angular velocity sensor) and an optical system is compensated for the movement according to the direction and the degree of the movement. In another conceivable method, the parallel moving extent of the whole image plane is detected through a video signal and is expressed in a movement vector. Then, the optical system is compensated on the basis of the vector.
In accordance with the method of using the acceleration sensor, the size of the device becomes larger thus requiring increases in space and weight. Besides, it results in a complex structural arrangement. This method is therefore hardly suited for a home video camera such as a camera-incorporating type video tape recorder which must be compact in size and light in weight.
As regards the method of computing and obtaining the movement vector of the image plane from the video signal, some camera movement that is intentionally caused by the operator might be mistakenly detected for a shake of the image. The device also would respond to a movement of the object which is in reality not a shake of the image. That method thus also has a serious problem.
The above-stated known image shake detecting devices include, for example, an image stabilizing camera which is disclosed in Japanese Laid-Open Patent Application No. SHO 61-248681. The camera of this kind is arranged as follows: an optical image is converted into an electrical signal by means of an image sensing system which consists of a lens system and a photo-electric converting element. A TV image signal is obtained through a signal processing operation performed in a given manner on the electrical signal by a signal processing circuit. The image signal thus obtained is supplied to an image shake detecting circuit as well as to a monitor. A correlation between two image planes obtained at a given time interval is detected from the image signal to find the degree and the direction of any image shake. Then, a driving control circuit and a motor are operated to control and move the lens system to offset the image shaking on the basis of the result of detection. The camera is thus arranged to be capable of obtaining a stable image even when the camera shakes.
The term "image stabilization" as used herein means a camera shake correcting function. The term "tracing" as used herein means a moving object tracing function of the camera. The processing and control operations are performed for these functions fundamentally in the same manner as mentioned in the foregoing.
However, the image shake detecting device which is arranged in this manner is incapable of making a discrimination between the movement of an object occurring only in a part of the image plane and a shake of the whole image plane. To solve this problem, the image shake detecting sensitivity of the device must be arranged to vary for the different areas of the image plane.
In detecting the quantity of shake from an image, the adoption of the method of detecting the quantity of shake with uniform sensitivity for the whole image plane does not permit a discrimination between a deformation or movement of the object and tilting of the camera by hand vibrations. In accordance with that method, the camera might trace a wrong object to take a picture of it against the intention of the photographer.
In connection with this problem, an image shake detecting device has been proposed as disclosed in an article entitled "About an Image Plane Shake Compensating Device", appeared in "The Technical Report" of the Television Society, Vol. 11, No. 3, pp. 43 to 48, PPOE, '87-12 (May, 1987). In the case of that device, the whole image plane is divided into 140 blocks of areas. A shake detection switch is arbitrarily turned on or off for each of these areas and an image shake is detected only from the areas for which the shake detection switch is turned on in accordance with a representing point matching method.
In accordance with the arrangement of this image shake detecting device, however, an image to be used as reference must be temporarily stored at a frame memory with its varied density values kept intact. To meet this requirement, the device necessitates the use of an analog-to-digital (hereinafter referred to as A/D) converter and a memory of a relatively large capacity. In addition to this shortcoming, the device is arranged to have one image plane superimposed on another by staggering them to a certain degree of vector and to find out a vector that gives the highest degree of coincidence. Therefore, the operation of the device includes a large amount of computation. The device thus necessitates a circuit arrangement on a large scale and requires a long computing time.
Besides, it has been extremely difficult to have the image shake detecting device incorporated in a compact video camera which must be capable of carrying out real-time processing and must be arranged in a compact circuit arrangement, because, the above-stated device of the prior art necessitates the use of a large circuit arrangement including the A/D converter, the frame memory, a computing circuit, etc., and a long processing time.
An image shake detecting area has been determined by the following methods: (1) A method of using a difference in luminance between the background and the object and (2) a method of detecting a movement occurring area from a difference occurring between the image of one frame and that of another obtained during a given period of time. However, in accordance with these methods, it is hardly possible to accurately determine the area in cases where there is no distinct difference in luminance between the background and the object (with the method (1)) and where a plurality of objects have moved (with the method (2)). In other words, each of the conventional methods has a demerit as well as a merit. Therefore, with the conventional image shake detecting device incorporated in a compact video camera, it does not enable the camera to accurately cope with every one of varied image conditions.
As regards the automatic focusing device for a video camera or the like, various automatic focusing devices of the type using the image signal of the video camera have been proposed. It is a feature of this type that an image sensor serves also as a sensor for automatic focusing. However, the following problem arises when the distance measuring area is broadened over the whole image plane: in cases where, for example, one object located at a far distance and another object located at a near distance coexist within one and the same image plane, the lens might be focused on one of them that has a stronger signal than the other even though the lens is intended to be focused on the other object. This is known as a "far-near competing condition". To avoid this, it has been practiced to limit the distance measuring area to a size about one quarter of the whole image plane and to have it always located in the center of the image plane, because of a high probability of having a desired (or main) object in the center of the image plane.
However, if the size of the distance measuring area is arranged to be unvarying, the image of the object might become too large or too small relative to the image plane. As a solution of this problem, there has been proposed a video camera that is arranged to have one of two or three different sizes of the distance measuring area manually selectable. Despite such arrangement, however, the possibility of an inappropriate size of the area still remains. The so-called "far-near competing condition" would take place if the area is too large. If the distance measuring area is too small, the main object tends to come out of the area to cause the camera focused on the background instead of the object when either the object is moving or the camera is shaking. Therefore, this problem also still remains to be solved.
Further, there has been proposed an apparatus which uses an image shake detecting device for objects appearing within an image plane and is arranged to trace (track) a moving object and to continuously perform control for accurate automatic focusing and accurate automatic exposure control. For example, apparatuses of this kind have been disclosed in U.S. patent applications Ser. Nos. 737,163 filed on May 23, 1985; 106,427 filed on Oct. 8, 1987; 154,078 filed on Feb. 9, 1988; 237,511 filed on Aug. 26, 1988; 240,915 filed on Sep. 6, 1988; 258,692 filed on Oct. 17, 1988; and 264,204 filed on Oct. 28, 1988. However, as mentioned in the foregoing, it is difficult to accurately make a discrimination between the movement of only one object in a part of the image plane and the movement of the whole image plane due to the movement of the camera. In accordance with the above-stated method disclosed in Japanese Laid-Open Patent Application No. SHO 61-248681, the degree of accuracy would lower if there is no luminance difference because the method utilizes a difference in luminance between the background and the object to be photographed. Further, in accordance with the technique disclosed in the above-stated Technical Report of the Television Society, it becomes difficult to accurately determine the areas in a case where a plurality of objects come to move. In other words, each of the methods of the prior art has a disadvantage as well as an advantage. Therefore, with the prior art image shake detecting devices incorporated in a compact video camera in accordance with these methods, they are incapable of adequately coping with all image condition.
With respect to the image shake detecting device of this kind, further examples have been disclosed in U.S. patent application Ser. No. 855,732 filed on Apr. 25, 1986, U.S. patent application Ser. No. 880,152 filed on Jun. 30;, 1986, etc. These patent applications disclose an image stabilizing camera which is arranged to compensate the optical axis of the lens system by detecting an image shake through the edge component of the image of an object to be photographed; and an arrangement to recognize an image through a histogram which represents the distribution of the feature of the image.